Readers’ cognitive processes during IELTS reading tests: evidence from eye tracking

The research described in this report investigates readers' mental processes as they complete onscreen IELTS (International English Language Testing System) reading test items. It employs up-to-date eye tracking technology to research readers' eye movements and aims, among other things, to contribute to an understanding of the cognitive validity of reading test items (Glaser. 1991; Field forthcoming). Participants were a group of Malaysian undergraduates (n=71) taking an onscreen test consisting of two IELTS reading passages with a total of 11 test items. The eye movements of a random sample of these participants (n=38) were tracked. Questionnaire and stimulated recall interview data were also collected, and were important in order to interpret and explain the eye tracking data. Findings demonstrated significant differences between successful and unsuccessful test-takers on a number of dimensions, including their ability to read expeditiously (Khalifa and Weir. 2009). and their focus on particular aspects of the test items and the reading texts. This demonstrates the potential of eye tracking, in combination with post- hoc interview and questionnaire data, to offer new insights into the cognitive processes of successful and unsuccessful candidates in a reading test. It also gives unprecedented insights into the cognitive processing of successful and unsuccessful readers doing language tests. As a consequence, the findings should be of value to teachers and learners, and also to examination boards seeking to validate and prepare reading tests, as well as psycholinguists and others interested in the cognitive processes of readers

Researching participants taking IELTS Academic Writing Task 2 (AWT2) in paper mode and in computer mode in terms of score equivalence, cognitive validity and other factors

Computer-based (CB) assessment is becoming more common in most university disciplines, and international language testing bodies now routinely use computers for many areas of English language assessment. Given that, in the near future, IELTS also will need to move towards offering CB options alongside traditional paper-based (PB) modes, the research reported here prepares for that possibility, building on research carried out some years ago which investigated the statistical comparability of the IELTS writing test between the two delivery modes, and offering a fresh look at the relevant issues. By means of questionnaire and interviews, the current study investigates the extent to which 153 test-takers’ cognitive processes, while completing IELTS Academic Writing in PB mode and in CB mode, compare with the real-world cognitive processes of students completing academic writing at university. A major contribution of our study is its use – for the first time in the academic literature – of data from research into cognitive processes within real-world academic settings as a comparison with cognitive processing during academic writing under test conditions. The most important conclusion from the study is that according to the 5-facet MFRM analysis, there were no significant differences in the scores awarded by two independent raters for candidates’ performances on the tests taken under two conditions, one paper-and-pencil and the other computer. Regarding analytic scores criteria, the differences in three areas (i.e. Task Achievement, Coherence and Cohesion, and Grammatical Range and Accuracy) were not significant, but the difference reported in Lexical Resources was significant, if slight. In summary, the difference of scores between the two modes is at an acceptable level. With respect to the cognitive processes students employ in performing under the two conditions of the test, results of the Cognitive Process Questionnaire (CPQ) survey indicate a similar pattern between the cognitive processes involved in writing on a computer and writing with paper-and-pencil. There were no noticeable major differences in the general tendency of the mean of each questionnaire item reported on the two test modes. In summary, the cognitive processes were employed in a similar fashion under the two delivery conditions. Based on the interview data (n=30), it appears that the participants reported using most of the processes in a similar way between the two modes. Nevertheless, a few potential differences indicated by the interview data might be worth further investigation in future studies. The Computer Familiarity Questionnaire survey shows that these students in general are familiar with computer usage and their overall reactions towards working with a computer are positive. Multiple regression analysis, used to find out if computer familiarity had any effect on students’ performances on the two modes, suggested that test-takers who do not have a suitable familiarity profile might perform slightly worse than those who do, in computer mode. In summary, the research offered in this report offers a unique comparison with realworld academic writing, and presents a significant contribution to the research base which IELTS and comparable international testing bodies will need to consider, if they are to introduce CB test versions in future

Researching mobile-assisted Chinese-character learning strategies among adult distance learners

In the field of teaching and learning Chinese as a foreign language (CFL), most studies investigate Chinese character learning strategies in pen-and-paper study by campus-based students. With the increase in distance-learning, and expanding popularity of smartphones and tablets and widespread availability of mobile applications for language learning, it is now important to research mobile-assisted character learning strategies by distance learners. This study attempts to i) identify changing practices when character learning is assisted by mobile technology; and ii) analyse the underlying causes for the changing practices. These issues were investigated using an online survey, learning diary and email interviews with UK-based distance learners of beginners’ Chinese (n=22). The findings suggest that there are important emerging new practices and previously unreported strategies such as constantly testing oneself and using pinyin input to identify new characters. They also reveal two strategy types not reported in previous research concerning social and affective dimensions. The study thereby provides important new insights into how adult distance learners of Chinese at beginners’ level learn characters when assisted by mobile technology, with significant pedagogical implications for Chinese character pedagogy in terms of course design and learner support

Researching metadiscourse markers in candidates’ writing at Cambridge FCE, CAE and CPE levels

This paper reports on research funded through the Cambridge ESOL Funded Research Programme, Round Three, 2012

Researching L2 writers’ use of metadiscourse markers at intermediate and advanced levels

Metadiscourse markers refer to aspects of text organisation or indicate a writer’s stance towards the text’s content or towards the reader (Hyland, 2004:109). The CEFR (Council of Europe, 2001) indicates that one of the key areas of development anticipated between levels B2 and C1 is an increasing variety of discourse markers and growing acknowledgement of the intended audience by learners. This study represents the first large-scale project of the metadiscourse of general second language learner writing, through the analysis of 281 metadiscourse markers in 13 categories, from 900 exam scripts at CEFR B2-C2 levels. The study employed the online text analysis tool Text Inspector (Bax, 2012), in conjunction with human analysts. The findings revealed that higher level writers used fewer metadiscourse markers than lower level writers, but used a significantly wider range of 8 of the 13 classes of markers. The study also demonstrated the crucial importance of analysing not only the behaviour of whole classes of metadiscourse items but also the individual items themselves. The findings are of potential interest to those involved in the development of assessment scales at different levels of the CEFR, or to teachers interested in aiding the development of learners

Using eye-tracking research to inform language test validity and design

This paper reports on a recent study which used eye-tracking methodology to examine the cognitive validity of two level-specific English Proficiency Reading Tests (CEFR B2 and C1). Using a mixed-methods approach, the study investigated test takers’ reading patterns on six item types using eye-tracking, a self-report checklist and stimulated recall interviews. Twenty L2 participants completed 30 items on a computer, with the Tobii X2 Eye Tracker recording their eye movements on screen. Immediately after they had completed each item type, they reported their reading processes by using a Reading Process Checklist. Eight students further participated in a stimulated recall interview while viewing video footage of their gaze patterns on the test. The findings indicate (1) the range of cognitive processes elicited by different reading item types at the two levels; and (2) the differences between stronger and weaker test takers' reading patterns on each item type. The implications of this study to reflect on some fundamental questions regarding the use of eye-tracking in language research are discussed. The paper concludes with recommendations for future research in these areas

Researching lexical thresholds and lexical profiles across the common European framework of reference for languages (CEFR) levels assessed in the Aptis test

This project uses automated analysis software (www.textinspector.com) to research the lexical and metadiscourse thresholds, and lexical and metadiscourse profiles, of test-takers’ writing in the British Council's Aptis Writing test, benchmarked to the Common European Framework of Reference for Languages (CEFR). Large quantities of Aptis writing responses (n=6,407), representing 65 countries, together with their score data, were analysed in terms of their use of lexis and metadiscourse. Measures and datasets used in the analysis include standard readability measures, the British National Corpus, the Corpus of Contemporary American English, English Vocabulary profile, the Academic Word List, and a bespoke corpus of metadiscourse markers. The purpose of the research is to enhance the validation argument for the Aptis test through large-scale profiling of candidates’ writing performance. The findings reveal that the Aptis writing test provides evidence that lexical complexity changes systematically as the CEFR level of learners increases. Of the 110 Text Inspector metrics used in the study, 26 metrics were significant across all CEFR boundaries, including measures of text length (sentence, token and type count), and metrics of lexical sophistication (syllable count and number of words with more than two syllables). Fourteen of the 26 metrics represent vocabulary use. One metric of text complexity (voc-d) was also significant across all thresholds. The study also explores the utility of these metrics for use in an automated scoring engine. Twenty metrics were used to build an ordinal logistic regression which was trained on a stratified subset of the data. This model was then used to predict the CEFR band of a testing subset which held nationality data constant. The data revealed that lexical use metrics from the Cambridge Learner Corpus (CLC) were the most successful at identifying CEFR level, and the model was most successful in identifying A1 and C-level responses. However, the model failed to accurately differentiate A2, B1 and B2 responses, suggesting that other, organisational variables play a significant role in human judgements, which are not accounted for in this study. The paper concludes with recommendations for rater training on the basis of the findings

Cell-matrix biology in vascular tissue engineering

We are developing biocompatible small-calibre vascular substitutes based on polymeric scaffolds that incorporate cell-matrix signals to enhance vascular cell attachment and function. Our graft scaffold comprises an outer electrostatically spun porous polyurethane layer seeded with smooth muscle cells, and a luminal polycaprolactone layer for endothelial cell attachment. Vascular cell adhesion properties of three vascular elastic fibre molecules, tropoelastin, fibrillin-1 and fibulin-5, have been defined, and adhesion fragments optimized. These fragments are being used to coat the scaffolds to enhance luminal endothelial cell attachment, and to regulate smooth muscle cell attachment and function. Tropoelastin-based cell seeding materials are also being developed. In this way, vascular cell-matrix biology is enhancing graft design

Anti-IL5 therapies for asthma

Background This review is the first update of a previously published review in The Cochrane Library (Issue 7, 2015). Interleukin-5 (IL-5) is the main cytokine involved in the activation of eosinophils, which cause airway inflammation and are a classic feature of asthma. Monoclonal antibodies targeting IL-5 or its receptor (IL-5R) have been developed, with recent studies suggesting that they reduce asthma exacerbations, improve health-related quality of life (HRQoL) and lung function. These are being incorporated into asthma guidelines. Objectives To compare the effects of therapies targeting IL-5 signalling (anti-IL-5 or anti-IL-5Rα) with placebo on exacerbations, health-related qualify of life (HRQoL) measures, and lung function in adults and children with chronic asthma, and specifically in those with eosinophilic asthma refractory to existing treatments. Search methods We searched the Cochrane Airways Trials Register, clinical trials registries, manufacturers' websites, and reference lists of included studies. The most recent search was March 2017. Selection criteria We included randomised controlled trials comparing mepolizumab, reslizumab and benralizumab versus placebo in adults and children with asthma. Data collection and analysis Two authors independently extracted data and analysed outcomes using a random-effects model. We used standard methods expected by Cochrane. Main results Thirteen studies on 6000 participants met the inclusion criteria. Four used mepolizumab, four used reslizumab, and five used benralizumab. One study in benralizumab was terminated early due to sponsor decision and contributed no data. The studies were predominantly on people with severe eosinophilic asthma, which was similarly but variably defined. Eight included children over 12 years but these results were not reported separately. We deemed the risk of bias to be low, with all studies contributing data being of robust methodology. We considered the quality of the evidence for all comparisons to be high overall using the GRADE scheme, with the exception of intravenous mepolizumab because this is not currently a licensed delivery route. All of the anti-IL-5 treatments assessed reduced rates of 'clinically significant' asthma exacerbation (defined by treatment with systemic corticosteroids for three days or more) by approximately half in participants with severe eosinophilic asthma on standard of care (at least medium-dose inhaled corticosteroids (ICS)) with poorly controlled disease (either two or more exacerbations in the preceding year or Asthma Control Questionnaire (ACQ) 1.5 or more). Non-eosinophilic participants treated with benralizumab also showed a significant reduction in exacerbation rates, but no data were available for non-eosinophilic participants, and mepolizumab or reslizumab. We saw modest improvements in validated HRQoL scores with all anti-IL-5 agents in severe eosinophilic asthma. However these did not exceed the minimum clinically important difference for ACQ and Asthma Quality of Life Questionnaire (AQLQ), with St. George's Respiratory Questionnaire (SGRQ) only assessed in two studies. The improvement in HRQoL scores in non-eosinophilic participants treated with benralizumab, the only intervention for which data were available in this subset, was not statistically significant, but the test for subgroup difference was negative. All anti-IL-5 treatments produced a small but statistically significant improvement in mean pre-bronchodilator forced expiratory flow in one second (FEV1) of between 0.08 L and 0.11 L. There were no excess serious adverse events with any anti-IL-5 treatment, and indeed a reduction in favour of mepolizumab that could be due to a beneficial effect on asthma-related serious adverse events. There was no difference compared to placebo in adverse events leading to discontinuation with mepolizumab or reslizumab, but significantly more discontinued benralizumab than placebo, although the absolute numbers were small (36/1599 benralizumab versus 9/998 placebo). Mepolizumab, reslizumab and benralizumab all markedly reduced blood eosinophils, but benralizumab resulted in almost complete depletion, whereas a small number remained with mepolizumab and reslizumab. The implications for efficacy and/or adverse events are unclear. Authors' conclusions Overall our study supports the use of anti-IL-5 treatments as an adjunct to standard of care in people with severe eosinophilic asthma and poor control. These treatments roughly halve the rate of asthma exacerbations in this population. There is limited evidence for improved HRQoL scores and lung function, which may not meet clinically detectable levels. There were no safety concerns regarding mepolizumab or reslizumab, and no excess serious adverse events with benralizumab, although there remains a question over adverse events significant enough to prompt discontinuation. Further research is needed on biomarkers for assessing treatment response, optimal duration and long-term effects of treatment, risk of relapse on withdrawal, non-eosinophilic patients, children (particularly under 12 years), and comparing anti-IL-5 treatments to each other and, in people eligible for both, to anti-immunoglobulin E. For benralizumab, future studies should closely monitor rates of adverse events prompting discontinuation

Vilanterol and fluticasone furoate for chronic asthma in adults and children

Background Vilanterol (VI) is a long-acting beta2-agonist (LABA) that binds to the beta2-adrenoceptor on the airway smooth muscle, producing bronchodilation. LABA therapy, which is well established in adults as part of the British Thoracic Society (BTS) Guidelines for the Management of Asthma, leads to improvement in symptoms and lung function and reduction in exacerbations. At present, the commonly used LABAs licensed for use in asthma management (formoterol and salmeterol) require twice-daily administration, whereas VI is a once-daily therapy. Fluticasone furoate (FF) is an inhaled corticosteroid (ICS), and ICS therapy is recommended by the BTS asthma guidelines. ICSs, the mainstay of asthma treatment, lead to a reduction in both airway inflammation and airway hyper-responsiveness. Regular use leads to improvement in symptoms and lung function. ICSs are currently recommended as 'preventer' therapy for patients who use a 'reliever' medication (e.g. short-acting beta2 agonist (SABA), salbutamol) three or more times per week. Most of the commonly used ICS treatments are twice-daily medications, although two once-daily products are currently licensed (ciclesonide and mometasone). At the present time, only one once-daily ICS/LABA combination (FF/VI) is available, and several other combination inhalers are recommended for twice-daily administration